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1.  Introduction 

In  connection  with  problems  of  control  mechanisms,  there  have 
been  studied  systems  of  differential  equations  In  \dilch  some  more  or 
less  arbitrary  "control  functions"  appear.  An  Important  question  is 
how  to  choose  those  control  functions  in  order  to  minimize  (or  maximize) 
a  certain  functional  of  the  solution.  Physically  speaking,  this  may  mean 
to  operate  the  system  In  the  "best"  possible  way,  minimizing  a  "cost" 
function  or  simply  the  time  of  operation  [l]. 

Pontrjagln  and  his  collaborators  [2]  have  pointed  out  a  general 
principle  (the  "maxim\jm  principle"),  \rtiich  applies  to  this  problem. 

The  problem  of  what  can  be  accomplished  by  adequate  election  of 
the  control  function,  disregarding  the  optimization  problem,  has  been 
studied  independently  by  Hadman  [^]  and  the  author  [4],  [^]. 

In  this  paper  we  want  to  look  at  these  problems  from  a  geometrical 
point  of  view;  our  aim  is  to  show  some  interesting  relations  which  remain 
hidden  In  a  more  analytical  treatment,  like  that  of  Pontrjagln,  Bolty^skil 
and  Gamkrelldze . 

We  shall  study  the  system 


(1.1) 
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where,  as  usual  k  ■  dx/dt.  Let  ub  vnrlte 

(1.2)  x°  «  t,  =  1 

and  Introduce  the  following  vectors 

X  =  (x  ,  ...>  X  ) 

X  =  (x°,  x\  x")  *  (x°,  x) 

(1.3)  f=(f^...,  f^) 

f  =  (f°,  ...,  f")  -  (f°,  f) 

u  =  (u  ,  u  ). 

These  vectors  are  considered  In  euclidean  vector  spaces,  with  the  usual 
2  12 

norm:  ||x||,  =  E(x  )  .  Equation  (l.l)  becomes,  then, 

(1.4)  i  «  f(t,  X,  u) 
or 

(1.5)  X  =  f(x,  u). 

We  make  the  following  assumptions: 

l)  f (x,  u)  is  defined  in  X  x  U,  where  X  is  the  x-space  (the  half¬ 
space  corresponding  to  x°  =  t  »  0  could  also  be  chosen)  and  U  is  a 
compact  set  of  the  u-space. 
il)  f(x,  u)  is  continuous  in  (x,  u). 

Hi)  The  Lipschitz  condition  holds: 

Iff  (Xg,  u):  -  f  (xj^,  u)  I!  <  K*  llxg  -  Xj^ll . 
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iv)  ||f(x,  u)||  =  0(||x|j)  for  ||x||  -»  00.  This  assvunptton  is  made  in  order 
to  avoid  a  finite  escape-time  for  the  solutions. 

We  shall  consider  the  parameter  u  as  a  function  of  t.  u(t) 
will  be  called  an  "admissible  control  fimction"  if 


v)  u(t)  is  measurable, 

Vi)  for  each  t,  u(t)  e  U. 

vii)  "Convexity  condition"!  f(x,  U)  =  {f(x,  u) |u  e  u)  is  a  convex  set 
for  each  x. 


We  note  that  being  U  compact  and  f(x,  u)  continuous,  the  set 
f(x,  U)  is  compact  too. 

According  to  well  known  theorems  (see  [6],  [7])^  these  conditions 
guarantee  the  existence  and  uniqueness  of  the  solution  of  (I.5)  for  any 
admissible  control  function  u(t).  The  iraportance  of  the  convexity  condi¬ 
tion  (vii)  will  appear  in  the  theorem  stating  that  the  set  of  reachable 
points  is  a  closed  set. 


2.  Reachable  set. 

Given  the  equation  (1.5)^  we  shall  say  that  the  point  x^  is 

reachable  from  x  ,  if  there  exists  an  admissible  control  function  u(t) 
o 

defined  in  the  time  interval  (t  ,  t^ )  =  (x  ,  x  °)  where  t^  S  t  such 

o  1  o  1  1  o 

that  equation  (I.5),  where  u  -  u(t),  together  with  the  initial  condition 

x(t  )  =  X 
'  o'  o 

has  the  solution  x(t)  satisfying 

x(ti)  =  x^. 

The  set  of  all  points  x  which  are  reachable  from  x^,  will  be 

called  "reachable  set  from  x  ",  and  denoted  by  R  .  When  there  is  no 

■ _ ^  *0 

possibility  of  misunderstanding,  the  subindex  x^  will  be  dropped. 
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In  many  simple  cases  it  is  easy  to  see  which  points  are  reach¬ 
able  and  which  are  not;  the  linear  case  has  been  treated  in  detail  in 
[1],  [8].  Here  we  are  concerned  with  rather  general  properties  of  the 
reachable  set. 

Theorem  2.1;  If  the  equation  (1.5)  fulfills  the  conditions  (i)  to  (vll), 

then  for  any  given  initial  x  ,  the  reachable  set  R  is  a  closed  set. 

The  proof  of  this  theorem  is  given  in  [9].  A  similar  theorem 
under  less  general  conditions  is  proved  in  [10 j.  The  importance  of  this 
theorem  lies  in  the  fact  that  it  may  be  used  as  an  existence  theorem  for 
solutions  of  optimality  problems  like  those  vdiich  are  the  concern  of 
PontrJagln*s  Maximum  Principle. 

3 .  Some  properties  of  the  boundary  of  the  reachable  set . 

It  is  important  to  study  the  properties  of  the  boundary  in 

the  space  X  =  ((x°,  x\  because  there  we  might  expect  to 

find  the  solution  of  problems  of  the  variational  type.  Let  us  take 

three  examples. 

a)  Minimum  time  for  reaching  a  given  set  G  of  the  space 

X  =  ((x^,  x*^)). 

Let  us  start  at  x  =  (x°,  x^,  x"^),  where  x°  =  t  ;  we 

o  o'  '  o''  o  o' 

want  to  reach  any  point  x  =  (x°  x^,  x^),  such  that 

A  1  n  ^ 

x^  =  (x-,,  x^)  e  G  and  t^  =  x°  has  the  least  possible  value.  Let 

+  ^  +  Q 

G  be  the  cylindrical  set  I  X  G,  where  I  is  that  part  of  the  x  axis, 
for  whjch  t  g  t^.  Then,  what  we  are  looking  for  is  a  point  x^  e  R  D  G 
vrtilch  minimizes  x°  =  t^.  This  point  will  belong  to  the  boundary  of  R, 
because  in  every  neighborhood  of  it  there  are  points  of  G  with  smaller 
x^  coordinates,  which  cannot  belong  to  R  because  they  do  not  belong  to 
G  n  R. 

b)  If,  in  the  preceding  example,  instead  of  minimizing  the  time 
t^  =  x°,  we  want  to  minimize  some  fvinction  of  the  coordinates 
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♦(x°,  x")  then  some  additional  assumptions  have  to  he  made.  If 

on  the  set  G,  the  function  ♦(x)  has  some  mlnimm,  attained  at  the  set 
G*  C  G,  and  if  G*  (1  R  ♦  ( some  part  of  G*  is  reachable ) ,  then  einy 
point  of  G*  n  R  gives  us  a  solution  of  our  problem.  These  points  do  not 
need  to  belong  to  the  boundary  of  R,  and  this  case  is  called  "degenerate" 
by  Rozonoer  [11  ],  because  Pontrjagln's  principle  does  not  apply  to  it.  In 
fact  (as  we  shall  see),  this  principle  applies  if  x^  e  G*  D  R  belongs  to 
the  boundary  dR. 

c)  If  we  want  to  minimize  some  functional  of  the  type 
t 

(3.1)  /  cp(x(t))dt, 

t 

o 

the  classical  procedure  consists  in  adding  one  coordinate  to  the  X-space, 
putting 

x"""^  =  (j)(x)}  x"‘^^(0)  =  0, 

and  minimizing  i(x^)  =  x”  . 

In  the  example  (b)  it  is  also  possible  to  write  x^^^  =  i(x°,  x^, ...,  x*') 
11*^1 

and  minimize  x“  ,  but  in  that  case  R  is  contained  in  the  hypersurface 
x**  =  t(x°,  ...,  x”),  BO  that  R  coincides  with  its  boundary  and  those 

properties  which  depend  on  tte  fact  that  the  solution  point  belongs  to  SR, 
become  trivial. 

In  this  case,  the  value  of  the  integral  (5.1)  depend  in  general  on 
the  trajectory  x(t),  so  that  the  set  R  does  not  need  to  be  contained 
in  some  hypersurface.  If  we  want  to  reach  the  set  G  we  have,  therefore, 
to  look  for  the  points  on  G  fl  R  with  mlnimiun  These  are,  of  course, 

on  the  boimdary  of  R  because  the  set  G  is  cylindrical  with  respect  to 
the  coordinate 

We  now  pass  to  establish  a  basic  property  of  the  trajectories  be¬ 
longing  to  the  boundary  of  R, 
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Theorem  5.1;  If  fc"  equation  (I.5),  R  is  the  reachable  set  from  the 
point  I  ,  if  the  point  e  SR  and  if  Xj^(t)  (t^  S  t  S  t^)  is  a 
trajectory  going  frcm  x^(t^)  =  to  x^(t^)  =  which  corresponds 

to  an  admissible  control  Uj^(t)  (t^  S  t  ^  t^),  then  the  whole  trajectory 
x^(t)  (t^  ^  t  S  tj^)  belongs  to  the  boundary  of  R. 

The  fact  that  e  R  guarantees  the  existence  of  the  functions 
u^(t)  and  x^(t)  mentioned.  Let  xis  assume  that  for  t  =  t^, 

(t^  g  tg  S  t^),  the  point  Ig  =  5Cj^('t2)  interior  to  R.  There  exists, 
therefore,  a  neighborhood  of  1^,  contained  in  R.  Consider  the 

equation 

(3-2)  i  =  f(|,  u^(t)) 

in  the  interval  (t^,  t^)>  where  now  the  control  function  u^(t)  is 
fixed.  For  the  initial  condition  |(t2)  =  we  obtain 
Thus  there  is  a  certain  neighborhood  of  such  that  |(t^)  e 

implies  i(t^)  e  A^.  We  shall  now  prove  that  A^C  R#  >rtiicb  contradicts 
the  fact  that  e  3r. 

Let  be  any  point  of  A^.  Equation  (5.2)  together  with 

=  1]^  defines  a  trajectory  such  that  iCt^)  =  I2  ^  Since 

e  R,  there  exists  an  admissible  control  function  u^Ct)  defined  in 
the  interval  (t^,  t^)  (where  t^  =  1^°),  such  that 

(3.3)  X2(t)  =  f(x2(t),  U2(t));  Xg(t^)  = 

gives  X2(tp  =  1^. 

We  now  extend  this  trajectory,  coupling  it  with  the  path  given  by 
(3.2),  in  the  following  manner; 

U2(t)  =  ^ 

*2^^)  =  UgCt));  X2(tp  = 


t  ’  S  t 


.t- 


(3.4) 


As  this  trajectory  coincides  with  the  former  g(t)  (  ve  have  shifted  the 
t  hut  not  the  x.valuB8),  we  obtain  b  (see  Figiire  l).  Hence, 

e  R  u  ye  wanted  to  prove  . 


Figure  1 

The  variational  and  the  adjol^it  equation. 

It  Is  well  known  that,  for  a  fixed  control  function  u*(t)  and 
the  corresponding  solution  x*(t)  of  equation  (I.5),  which  we  suppose 
given  in  the  time  interval  (t^,  t^),  the  neighboring  solutions  of  (I.5) 
can  be  represented  by  the  variational  equation 

(5.5)  y  “  u  =  u*(tr  y  • 

X  =  x*(t) 


Here 

(5.6)  y(t)  =  x(t)  -  x«(t)  +  o(||x(t^)  -  x*(t^)||)  (•) 

Is  the  first  variation  of  x*(t)  and  Is  the  Jacobian  matrix. 

Since  the  equation  (3.5)  is  linear,  Its  solution  is  Of  the  form 

(3.7)  y(t)  =  A{t,  t^)'y(tQ), 

where  A(t,  t^)  Is  a  linear  operator  (i.e,,  a  matrix). 

^  ^  y  =  o  (x)  ^means  that  11m  ^  =  0. 
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The  y  Is  a  contravarlant  (column)  vector,  of  the  tangent  space > 
and  if  p  =  p(t)  is  a  covariant  (row)  vector  such  that 

(5.8)  p(‘t)*y(t)  =  const.. 

It  Is  known  that  p(t)  satisfies  the  adjoint  equation 

(5-9)  i»  =  -P  •  [^1  ^  ^  , 

X  =  x*(t) 


the  solution  of  which  is 

(3.10)  p(t)  =  p(t^)  •  A(t^,  t). 

This  vector  p  is  very  useful  for  defining  hyperplanes  in  the 
tangent  space,  as  we  shall  see. 

Theorem  3.2;  If,  for  equation  (1.5)^  R  is  the  reachable  set  from  a 
certain  point  |^,  if  x*(t),  (t^^  S  t  S  t^)  is  an  admissible  trajec¬ 

tory  of  (1.5)  belonging  to  the  boundary  of  R  and  if  at  the  points  of 
x*(t)  the  boundary  of  R  is  a  smooth  surface  with  a  tangent  plane  given 
ty  p(t)’(x  -  x*(t))  =  0,  then  p(t)  satisfies  eq\jatlon  (3*9)* 

Proof;  Let  x(t)  (t^  ^  t  g  tg)  be  a  neighboring  trajectory  of  x*(t), 
which  corresponds  to  the  same  control  function  u*(t).  If  x(t^)  €  R 
then  evidently  x(t)  e  R  for  any  t  €  (t^,  tg).  !Bie  fact  that  p*(x  -  x*) 
is  the  tangent  plane  means  that 

p*(x  -  X*)  +  o(||x  -  x*||)  i  0  for  X  €  R 

cuid 


p-(x  -  X*)  +  odix  -  x*||}  S  0 


for  X  /  R 
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vhere  we  have  supposed,  for  example,  that  the  set  R  is  on  that  side 
of  the  tangent  plane  where  the  scalar  product  is  positive.  As  o(l|x  -  x*||) 
refers  to  x  -» x*,  we  are  interested  in  a  first  order  approximation  and 
may  substitute  x  -  x*  by  y  as  given  in  (5- 6).  So,  if  x(tj^)  €  R  and 
y(tj_)  =  x(t^)  -  x*(t^),  we  have 

(3.11)  p(t)-y(t)  +  o(l|y(t^)|l)  a  G  (t^  S  t  S  t^) 

and  this  determines  con^iletely  the  position  of  the  tangent  plane  given  by 
p(t),  since  the  mapping  y(t^)  -»y(t)  is  linear  and  transforms  a  half 
space  into  a  half  space.  Besides,  the  vector  p*(t)  defined  by  equation  (5*9) 
and  the  initial  value  p*(tj,)  =  p(tj^)  also  satisfies  the  relation  (5»ll) 
becauae  p*(t)*y(t)  =  const.,  so  that  identically  p*(t)  =  p(t). 


U.  »PBngent  cone  to  a  point  set. 

In  order  to  sinqilify  the  statement  of  the  properties  of  a  reachable 
set,  we  Introduce  the  concept  of  tauigent  cone  to  a  given  point  set  and 
start  proving  some  elementary  relations. 


Definition  4.1;  Given  a  point  set  A  of  the  space  X  and  a  point 
x^  e  X,  we  shall  call  tangent  cone  of  the  set  A  at  the  point  x^,  the 
set  of  vectors  y  such  that  there  exist  a  sequence  of  positive  numbers 
c^(l  =  1,2, 3i  •••)  and  a  sequence  of  points  x^  e  A  such  that 


(4.1) 


lim 
1  -4  oo 


=  X 


11m  Cj^(xj_  -  x^j)  =  y. 
1  -» oo 


We  shall  denote  the  tangent  cone  of  the  set  A  at  the  point  x^ 

by  T  [A],  or  simply  by  T[A1  when  there  is  no  possibility  of  mlsunder- 
*0 

standing. 
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According  to  the  definition,  if  is  an  exterior  point  of  the 

set  A,  the  cone  T  [A]  will  be  empty.  If  x  is  an  interior  point 

of  A,  it  will  be  the  whole  tangent  space  (all  vectors  y).  If  the  set 

A  is  a,  differentiable  r-dimensional  hypersurface  and  x^  €  A,  then  the 

cone  T  [A]  is  the  r-dimensional  tangent  linear  spacer 
*0 

Lemma  4.1;  If  A  =  A^  U  Ag,  then 
T[Al  =  T[Aj^]  U  T[Ag]. 

Proof ;  Obviously,  if  A^C  A,  also  C  T[A].  Therefore 

T[A]  2t[A^]  U  TlAg]. 

For  proving  the  opposite  relation,  it  is  sufficient  to  see  that  if 
y  e  T(Aj^  U  Ag],  there  exist  c^,  x^  such  that 

y  =  lim  c^(xj^  -  x^) 

with  Xj^  €  A^  U  Ag,  then  some  infinite  subsequence  x^^  belongs  to  one 
or  both  sets  A^  or  Ag.  ^ 

Lemma  4.2;  The  tangent  cone  of  emy  set  Is  closed. 

Proof;  We  have  to  show  that  if 

(4.2)  y  =  11m  y 

i  -»  oo''i 

and 

(4.5) 

where 


^1=  °iAj  “*o)  1  = 

J  -♦  oo 


■11- 


(4.4): 


cij>0;  x^j€A; 


lim 
J  -» oo 


X 


0 


> 


then  there  exist  some  seqiiences 

>  0>  €  Ky  CC  =  Xy  Zy  ^t  •  •  • 


such  that 


(4.5) 


lim  x^ 
a  ^  oo 


11m 
a  oo 


=  y- 


We  shall  construct  the  seqxience  x^  In  the  following  way.  By 
virtue  of  (4.3)  and  (4.4),  for  each  positive  Integer  a  we  can  select  a 
value  of  J,  which  we  call  such  that  both  Inequalities 


< 


1 

a 


and 


lie  ,  (x  .  X  ) 

“Ja  “Jo  “ 


hold.  Now  we  put 


a  rt  aj^ 

Obviously,  the  first  of  the  inequalities  (4.5)  holds.  FOr  showing  that 
the  second  holds  too,  we  write 

^  KK  -  *o>  -  ^  il^a  " 

and  see  that  for  any  e  >  0,  we  have  ||y^  -  y||  <  |  and 
||7^(x^  -  x^)  -  7^11  <1  for  all  a  greater  than  some  N(e). 
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Lenuna  4.3:  For  any  set  A  (and  any  po^t  x^) 

T[aA]  C  T[A]. 

The  proof  is  aimllar  to  the  preceding  one.  Suppose  that  y  e  T[SA] 
There  exist  sequences  c^  >  0  and  -» x^  such  that  "  *q)  -►  F* 

Furthermore,  as  x^^  €  S  A,  we  may  write 

X.  =  11m  X  ,  X  e  A. 

j  -» oo 


For  each  positive  Integer  a  we  can  select  a  value  of  J  =  Such  that 


-  X  If  =  o( 

a 


Hence 


and 


11m 

a  -♦  oo 


llm 
a  -» oo 


0 


llm 

a  -» oo 


c  (x  , 
a'  aj 


a 


-  X  )  =  llm 
o' 

a  -♦  oo 


c  (x 
a'  a 


this  last  relation  being  valid  because  x  .  -  x  =  (x  ,  -  x  )  +  (x  -  x  ) 

aj^  o  '  aj^  a'  '  a  o' 

Is  eui  Infinitesimal  which  Is  equivalent  to  x  -  x  .  So,  since  In  the 

a  o 

last  equality,  x  .  e  A,  we  conclude  that  y  e  T[A]. 

Corollary:  T[S]  =  T[A)>  where  5  denotes  the  closure  of  A. 


T[A]  =  T[A  U  dA];  =  T[A]:  U  T[^A]  =  T[A]. 


Proof; 


Theorem  4.l!  For  any  set  A 
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5T[A]C  T[aA]. 

We  start  proving  an  auxiliary  lemma ^ 

Lemma  U.4;  If  is  eoi  open  cone  (a  cone  projecting  an  open  set), 
if  A  la  a  given  set  and  if  is  a  non -zero  vector  fulfilling  the 
foUovlng  conditions: 


^0  ^  ^o 

11)  y  e  T  [A] 
o 

then,  for  any  e  >  0  -ttiere  exists  a  point  x  e  A  such  that  0  <  llx-x  j|  £  e 

II  Q  II 

and  X  -  X  €  Y  . 

o  o 

Indeed,  since  y^  e  [A],  there  exists  a  sequence  x^^ 

Xj^  6  A,  ->  Yq#  whire  c^  >  0.  Therefore,  ||x^  -  x^||  it  e 

for  i  greater  than  some  N^.  Since  Y^  is  open,  -  x^)  €  Y^  for 

i  greater  than  some  Ng.  Taking  1  >  oax  (N^,  Ng),  x^  fulfills  the 
desired  requirements. 

Now  we  return  to  the  proof  of  Theorem  4.1.  It  is  easy  to  see 
that  9t[A]  is  a  cone,  l.e.,  that  if  y^  €  St[A]  then  the  idiole  ray 
{cy^|c  >  0)  belongs  to  St[A].  Let  \is  suppose  that 


(4.6)  y^  €  dT^  [A]; 

o 

if  we  show  that  y^  necessarily  be.1.ongs  to  T  [SA],  the  theorem  will  be 
proved.  ° 

We  tsLke  a  sequence  of  open  connected  cones  Y.  such  that 
oo  t 

^i+1  -  ^i  ^  ^i  single  ray  cy^,  (x  >  0).  We  may  take,  for 

"  ioll 

6^  >  G ,  0 ,  and  project  them  from  the  center  of  the  sphere . 


II  <  with 


example,  on  the  unit  sphere  the  sets  ty| 
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Let  us  call  now  the  Intersection  of  the  cone  with 
the  hall  of  radius  l.e., 

Y*  =  (y|y  e  Y^j  llyl|  <  e^)> 

where  always  8^  -»0  (see  Figure  2). 

Since  y  e  [A],  In  any  open  cone  Y^  there  exists  two  rays, 
one  belonging  to  T[A]  °  and  the  other  to  Its  complement  CT[A].  From 
the  preceding  lemma  we  deduce,  therefore,  that  In  any  nel^borhood  of  the 
first  ray  there  Is  a  point  x*/  x^,  sUch  that  x*  €  A,  x^  -  x^  e  YJ. 


Figure  2. 

On  the  second  ray  there  exists  some  point  x^  ^  x^  such  that 
x^  /  A  and  x^  -  x^  c  Y^  (indeed,  a  \rtiole  segment  of  this  ray,  start¬ 
ing  at  x^,  does  not  belong  to  A,  otherwise  this  ray  would  belong  to 
T[A}. )  Therefore,  the  set  Y^  -  (x^)  being  connected,  there  exists  a 
point  /  x^  such  that  x^^  e  dA,  x^^  "  *o  ^  ^  11*1  ^  ^1’ 

From  here  we  conclude  that 

l)  X.  -» X  as  1-4  oo: 

10 

11)  Oj(Xj  -  x^)  ir  w  take  Cj  .  .  ,J|  >  0. 

Remark.  It  Is  li^ortant  to  note  that  the  converse  Is  not  necessarily 
true,  that  Is, 


T[Sa1  not  necessarily  C  5TtA] 
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as  Bhown  in  ah  example  >  In  Figure  3 . 


Figure  5 


5.  The  reachable  cone. 

Definition;  Given  eqmtlon  (l.  5)  and  a  certain  point  x  ,  let  R  be 

*o 

the  reachable  set  from  x^;  we  shall  call  "reachable  cone  from  the  point 

x,  at  the  point  x",  and.  denote  by  c  [x]  or  simply  C[x]>  the  tan- 
w  X 

gent  cone  of  R  at  the  point  x: 

o 

(5.1)  °x  “  ^x^\  ^  ' 

O  0 

Taking  Into  accovint  that  is  closed  and  connected,  it  is  easily 

o 

seen  that  a  necessary  and  sufficient  condition  for  C  [x]  to  be  not 

^0 

empty  Is  that  x  e  R  . 

*o 

Theorem  ^.1;  Let  u(t)  be  an  admissible  control  fimctlon  of  equation 
(l. 5)^  defined  in  the  interval  (t^,  tg)  where  t^  <  tg,  x(t)  the 
corresponding  trajectory,  let  C[x(t)]  =  C  [x(t)]  defined  as  above 
for  some  point  x^,  and  let  A(t,  t^)  be  °the  matrix  defined  in  (3.7) 
as  solution  of  the  variational  equation  along  x(t);.  then 

(5.2)  CExCtg)]  >A(tg,  t^)C[x(t^)]. 

Proof;  Suppose  y  e  C[x(t^)],  so  that  there  exists  a  sequence'  x^  -♦x(t^), 
x^  e  R,  -  x(t^))  =  Xj,  -»  >  0,  If  we  show  that 

A(t2,  t^).y  e  C[x;(tg)],  we  will  have  proved  the  theorem, 
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Let  x|  "be  the  point  obtained  by  translation  of  along  the 

trajectory  corresponding  to  the  same  control  function  u(t)  during  the 
time  interval  (t^,  tg)  (similarly  as  in  the  proof  of  Theorem  3.1).  Let 
yj^  be  the  value  for  t  =  tg  of  the  solution  of  the  variational  equation 
starting  at  y^^  for  t  =  t^,  so  that  (see  Figure  U) 

(5-5)  V[  =  A(t2,  t^)y^. 


Between  x*  and 


exists  the  relation 


^(tg)  =  ^  +  o(||x^  -  x(t^)ll). 


Figure  4. 

Therefore,  8iq;>po8ing  the  existence  of  these  limits, 

(5.14.)  y'  =  lim  y^  =  lim  c^(x*  -x  (tg)). 

Since  the  linear  operator  A(tg,  t^)  is  continuous,  we  obtain  from  (5.3) 

y'  =  lim  y»  =  A(tg,  t^)lim  y^^  =  A(tg,  t^)y 

which,  by  the  way,  proves  the  existence  of  the  limit.  Moreover,  from  the 
way  they  are  obtained,  it  follows  that  xj^  e  R,  hence  (5.4)  proves  that 
A(tg,  t^)y  e  C[x(tg)],  which  was  our  task. 

Theorem  5.2;  If  x(t)  is  the  trajectory  of  (I.5)  corresponding  to  the 
admissible  control  function  u(t),  defined  in  the  interval  (t^,  t^), 
if  x(tQ)  =  x^,  x(t^)  =  x^;  and  if  at  the  point  x^  the  tangent  cone 
C  [x, ]  is  contained  in  the  half-space  defined  by  p.  •  y  2  0,  where 

Xq  1  1 
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is  a  covariant  (row-)vector,  then,  at  any  point  x(t)  with 

t^  S  t  S  t^,  the  cone  C  [x(t )  ]  la  contained  In  the  half -space 

*o 

p(t)*y  ^  0  where  p(t)  *  Pj^*A(t^,  t)  Is  the  solution  of  the  adjoint  varia¬ 
tional  equation. 

Proof;  If  for  some  e  C[x(t)]  It  would  be  p(t)*y^  »  Pj^*A(tj^,  t)y^  <  0, 
applying  the  preceding  theorem  we  see  that 

^1  ”  ^^^t  ^  ^^*1^ 

and  <  0>  contradicting  ovir  hypothesis. 

This  theorem  generalizes  the  theorem  (5.2),  and  here  the  advantage 
Is  that  we  are  not  assuming  any  kind  of  smoothness  of  the  boundary  of 


Corollary  5.1;  If,  under  the  conditions  of  the  last  theorem,  the 

reachable  set  R  is  "flat"  In  the  neighborhood  of  the  point  x^,  this 
0 

Is  if  [x^]  is  contained  in  an  r-dlmenBlonal  linear  subspace,  the  same 
*0 

happens  along  the  trajectory  x(t)  (t^  S  t  S  t^). 

Corollary  5.2;  If,  under  the  seune  conditions  C  [x^  ]  has  a  con- 
yex  edge  or,  more  generally,  a  convex  vertex; 

p^*y  i  0,  ...,  p^  y  s  0;  y  ^  [*2^ I# 

*0 

the  same  happens  along  the  whole  trajectory  x(t)  (t^  ^  t  S  t^). 

When  C^  [x]  Is  convex  for  all  x,  as  In  the  linear  case,  we 
o 

obtain  the  result  that  any  trajectory  on  the  boundary  of  R  cannot  move 

o 

from  a  smooth  to  a  "rougher"  part  of  SR  (it  can  move  from  a  vertex  to  an 
edge  awl  from  there  to  a  region  where  SR  has  a  tangent  plane,  but  never 
in  the  opposite  direction). 
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6.  The  admissible  cone. 

Definition !  Given  eqmtlon  (I.5)  the  set  U  of  admlssihle  values 

of  the  variable  u,  vre  shall  call  "admissible  cone  at  point  x^",  and  de 

note  by  P[x  ],  the  cone  which  projects  the  set  f(x.  U);  this  Is 
O  P 

y  €  FtXg] 

11  emd  only  if  there  exists  a  value  e  U  and  a  positive  number  c 
such  that 

(6.1)  y  =  u^)* 

F[x]  is  closed  and  convex,  since  the  set  f(x,  U)  has  those  pro 

pertles. 

Note  that  f(x,  U)  is  the  section  of  the  cone  F[x]  with  the 
plane  y°  =  1. 

Our  task  now  is  to  study  the  relation  between  the  admissible  cone 
F[x]  and  the  reachable  set  R,  emd  especially  its  temgent  cone  T^[R]. 

Lemma  6.l!  If,  under  the  conditions  stated  in  Section  1,  u(t) 
is  an  admissible  control  function  and  x(t)  the  corresponding  trajectory 
of  (l.5)>  then 

(*•2)  .  i  '  f(x(t),  u{t))dt 

■  t 

o 

is  a  vector  belonging  to  the  convex  hull  of  the  set  (f(x,  U) |x  =  x(t); 

t  S  t  S  t  +  €), 
o  o 

This  is  a  vectorial  generalization  of  the  cleissical  mean  value 
theorem.  For  proving;  it  we  multiply  (6,2)  by  any  vector  p,  obtaining 
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P'  *  e  ^  °  u(t))dt. 

t 

o 

Now  the  integrand  Is  a  scalar  function,  so  that  we  have 

(6.3)  l.u.b.(p-f(x(t),  U))  i  p-  ^  i  g.l.b. (p-f(x(t),  U)) 

for  0  S  t  -  t^  S  e.  Since  these  inequalities  are  valid  for  any  vector 

p,  they  define  precisely  the  convex  hull  of  the  set  f(x(t),  U),  for 

t  S  t  S  t  +  €. 
o  o 

Lemma  6.2:  Let 

(6.4)  6  =  6(x^,  e)  =  l.u.b.(||f(x,  u)  -  f(x^,  u)l|} 
subject  to  the  conditions 

(6.4')  ||x  -  x^ll  s  €j  u  €  U. 

Then,  under  the  conditions  of  §1, 

(6.5)  €)  -+0  for  e  -*0. 

Proof:  As  8(x^,  c)  is  a  nondecreasing  function  of  e  for  e  >  0, 
there  exists  the  limit 


1  =  lim  6(x  ,  e). 

€  ^0  ° 

Let  us  take  the  sequences  of  positive  numbers  -»0,  -»0  for 

i  =  1,2,3,  ...  .  For  any  i  we  can  select  a  value  Uj^  e  U  and  a  value  x^ 

such  that  llx  -  x .  (I  ^  and 

0  1  1 

B(x^,  k  |lf(x^,  u^)  -  f(x^,  u^)||  g  B(x^,  f^)  -  p^. 
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As  i  -*  00;  -*  0  and  therefore 

||f(x^;  u^)  -  f(x^,  u^)||  -»/. 

According  to  (6.4*);  ^  compact;  we  can  select  a  sub- 

seq\ience  such  that  the  lim  exists;  we  shall  suppose  that 

i  =  1;2;5;  ...;  rofcrs  to  that  subsequence.  Hence;  the  continuity  of 
f(x;  u)  for  X  =  x^  assumes  that 

1  =  lim  i|f(x^;  u^)  -  f(x^;  u^)|l  =  0. 
i  -♦  oo 

With  these  elements  we  can  prove  for  rather  general  conditions, 

what  in  the  simplest  cases  is  quite  obvious,  namely  that  "In  first  approxi-^ 

mat ion"  the  reachable  points  from  a  given  x^,  are  those  which  are  inside 

the  cone  F[x  ]. 

o 

Theorem  6.1;  If  the  differential  equation  (1.5)  satisfies  the  conditions 
of  §1,  then 

(6.6) 

o  o 

Proof;  First  we  shall  prove  that  if  ye  F[x^],  then  /  ^  )•  This 

is  easy,  because  in  this  case  there  exists  a  positive  number  c  and  a 

value  u^  €  U  such  that  c*f(x^;  u^)  =  y,  and  if  we  integrate  the  eqmtlon 

(1.5)  with  u(t)  =  u^  =  const,  and  the  initial  condition  x(t^)  =  x^,  we 

obtain  a  trajectory  x(t)  belonging  evidently  to  R  ,  for  which  the 

*0 

vector  y  is  tangent  at  the  point  x  ,  and  therefore  y  g  T  [R  ]. 

*0  o 

For  proving  the  converse,  we  suppose  that  y  e  T„  [R  ].  Hehce,  there 

exist  sequences  c^  >  0,  x^  e  R^  (i  =  1,2,5,  •••)  such  thal 

o 

x^  =  lim  x;  y  =  lim  c^^Cx^  -  x^). 
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(6.7) 


The  relation  e  R  implies  the  existence  of  control  functions  u.*(t) 
i  x^  1  '  ' 

and  corresponding  trajectories  of  the  differential  equation  x^*(t),  lead¬ 
ing  from  x.*(t  )  =  X  to  x.*(t  +  ^4 )  =  X4.  >  0.  We  now  apply 

i  o  o  1  o  i  11  ■.-•r-v 

Leoiiia  6^1  to  the  equality 

t  +  ^ 

-  ^0=  f  °  u^*(t))dt4 

t 


^i  ~  ^o 

concluding  that  —  helongs  to  the  convex  hull  of 

(f(x,  U)  1  ||x  -  x^ll  <  T|^},  ^ere  =  ■n^(  might  be  taken  of  the  form 
q  =  K*i^,  taking  for  K  an  upper  bound  of  ||f(x,  u)||  in  a  suitable 

O/  .  M 

neighborhood  of  x  .  The  fact  that  the  component  f  [x,  u)  s  1  assures 
*1  “  ^o 

that  II - j - II  *  1  and  therefore  -♦  0  for  x.^ 

^  ^i  ~  ^o 

It  is  easy  to  see  that  the  sequence  — - -  contains  a  convergent 

subsequencej  in  fact,  it  is  contained  in  the  convex  hull  of 
(f  (x,  u)  I  ||x  -  x^ll  <  max  which  is  a  bounded  set.  Let  us  suppose  in 

the  following  that  the  indices  i  =  1,2, 3,  •••  refer  to  that  convergent 
subsequence . 

From  the  relation  of  Lemma  6.1  we  deduce  that  for  any  vector  p 


x,  -  X 


g.l.b.  (p.f(x,U)  I  ||x  -  x^il  <  q^)  S  p.  -7’^  -  <  l.u.b.{p.f  (x,U)  I  ||x-xj|:  <  q^^) 


this  following  from  the  fact  that 


g.l.b.  [p* convex  hull  {f(x,  U)  |  ||x  -  x^||  <  q^^}  ]  - 
=  g.l.b.  [p-  {f(xy  U)1  ||x  ~  xj\  <  q^)  ] 


and  similarly  for  the  l.u.b. 
We  may  write 
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l.Uib.  {p*f(x>  U)  |  ||x  -  xjl  <  T]^)  S  l.u.b.  (p-f(x^,  U))  + 

+  l.u.b,(p[f(x,  u)  ^  f(x^,  u)]  |u  e  U,  \\x  -  x^||  <  tj^) 

and 

g.l.b,  (P*f(x,  U)|  llx  -  x^ll  <  Ti^)  S  g.l.b.  {p*f(x^,  U))  - 

-  l.u.b.(p[f(x,  u)  -  f(x^,  u)]|  u  e  U,  ||x  -  x^||  <  tj^). 


By  Lemma  6.2,  the  last  term  in  both  inequalities  tends  to  zero  for  1  -» oo, 
therefore  taking  limits  in  (6.7)  we  get 


.8)  g.l.b. (p*f(x  ,  U))  ^  p*lim 


-  X 


Since  U))  Is  compact  and  convex  by  hypothesis,  and  (6.8) 


holds  for  any  vector  p. 


(6.9) 


11m  *1  "  *0 


e  f(x„,  U). 


II  i  P 


Besides,  since  —  li  *  1  and  bounded,  its  limit  is  neither  zero 

^  j 


nor  infinity,  therefore 

y  =  lim  c^(x^  -  x^)  =  (llm  c^i'j^)(llm  )  =  k  11m  ^  ^ 


X.  -  X 
1  0^ 


*1  ’•  *0 


which,  together  with  (6.9)  implies  that  y  €  F[x^]  by  definition  of  the 
admissible  cone. 

This  theorem  can  be  applied  to  compare  the  relative  position  of  the 
admissible  cone  F[x^]  at  any  other  point  than  x^.  Indeed,  x^  € 
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and  T  [R  ]  ^T  [R  ].  Applying  the  last  theorem 


Implies  R  C  R 

*1  ^0  *■!  *1  *1  *1 
ve  obtain  immediately  the  following  theorem 


Theorem  6.2;  If  the  differential  equation  (1.5)  satisfies  the  conditions 
of  §1,  said  the  point  is  reachable  from  then 

(6.10)  FEx^^IC  T  [R  1. 

1  o 


These  theorems  do  not  assert  anything  about  the  actual  reachability 
of  the  points  located  in  the  admissible  cone.  This  will  be  the  concern  of 
the  following  ones. 


Theorem  6.31  If,  under  the  assumptions  of  Theorem  6.1,  Y  is  a  closed  cone 
belonging  to  the  interior  of  the  cone  P[x^],  then  there  exists  eui  tj  >  0 
such  that  all  points  of  the  form 

(Xq  +  y  I  y  €  y,  ||y||  <  n) 

are  reachable  from  x  . 

o 

Proof;.  If  x(t)  is  the  solution  of  the  differential  equation  correspond¬ 
ing  to  the  control  function  u(t)  =  u  =  const.,  starting  at  x^>  we  may 
write 

II  ti(x(t),  u)  -  f(x^,  u)]dt||  s  K  /^°  ||x(t)  -  xj|dt  S 

0  o 

S  K  1°  dt  /  Iff (x(t),  u)|ldt  S  K  M 
'o  o 


where  K  and  M  are  suitable  constants  which  can  be  taken  independently 
of  u  for  all  u  €  U  (since  the  set  U  is  compact),  provided  t  is 
restricted  to  some  neighborhood  of  zero.  Therefore 


~2k~. 


.  t 

lim  -r  f  °  f(x(t),  u)dt  =  f(x  ,  u) 

t-»0  ^  t  ° 

o 

uniformly  in  u  e  U. 

The  set  f(x^,  U)  is  really  n-dimensional  (not  n+1  dimensional) 
because  its  zero  component  is  always  1.  We  shall  consider  in  >rtiat  follows 
the  n-dlmensional  Interior  of  this  set,  this  is  the  interior  according  to 
the  topology  of  the  n-dlmensional  hyperplane  f°  =  1  (see  fig.  5)» 

For  any  fixed  £  ,  the  mapping  of  n-space  into  n-space,  which  maps 
1  ^o'*'^ 

^(Xq,  u)  in  -  f(x(t),  u)dt  is  contlnvujus  in  its  argument  f(x  ,  u), 

o 

therefore  it  maps  the  compact  set  U)  onto  a  compact  set,  and  accord¬ 

ing  to  what  was  shown  before,  this  mapping  tends  \miformly  to  the  identity 
for  t  -*  0.  Taking  any  closed  set  Y’  belonging  to  the  n-^lmenslonal 
interior  of  f(xQ,  U),  there  exists  an  t)  >  0  such  that  for  every  positive 
£  <  T)  the  image  of  f(x^,  U)  in  the  above  defined  mapping,  covers  Y'; 
for  seeing  this  it  is  sufficient  to  take  tj  so  small  that  for  every  pair 
(£,  u),  0  <t<  T),  u  €  U  the  following  Inequality  holds: 

||f(x^,  u)  -  ~  f(x(t),  u)dt||  <  distance  from  set  Y' 

^  o  to  the  n-dlmensional 

boundary  of  f  (x  >  U)  . 


Fig.  5. 
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Now  we  take  as  set  Y*  the  Intersection  of  the  closed  cone  Y 
of  our  hypothesis  with  the  plane  (x  -  Xq)°  =  !•  Since  Y  Is  closed  and 
contained  In  the  (n+1  dimensional)  Interior  of  F[x^],  also  Y*  will  he 
closed  and  contained  In  the  (n-dlmenslonal)  Interior  of  f(x^,  U).  We  may, 
therefore,  apply  the  preceding  result  stating  that  there  exists  eui  t)  >  0 
such  that  for  any  y*  e  Y*  and  any  positive  t  <  there  exists  an  u  €  U 
such  that 

t  +{ 

y*  »  /  °  f(x(t),  u)dt. 
t 


We  shall  prove  that  this  tj  fulfills  the  requirement  of  the  theorem. 
Indeed,  If  y  e  Y  and  l|y||  <  t),  we  take  y*  =  y/y°  and  £  =  y°  S  ||y||  <  t) 
(y°  Is  the  zero  cooqionent  of  tne  vector  y,  which  Is  positive  because 
y  €  F[x^]).  Then  y*  e  ^(*q»  U)  and  we  may  apply  the  last  result,  obtain¬ 


ing  that 


t  +£ 

Xq  +  y  -  Xq  +  £  y'  =  x^  +  /  f(x(t),  u)dt 

^0 


for  some  value  of  u  c  U,  l.e.,  +  y  Is  actually  reachable  from  x^. 


Corollary  6.1;  If  R  Is  the  reachable  set  from  the  point,  x^,  Q  is 
the  complement  of  R  (l.e.,  the  set  of  not  reachable  points)  and  If  x^^ 

Is  reachable  from  x^,  then,  under  the  assiaaptlons  of  the  preceding  theorems, 
the  Intersection  F[x, j  n  T  tQ]  is  void  or  belongs  to  the  boiaidary  of 

X 


F[x^]. 


Indeed,  if  we  suppose  that  y  belongs  to  the  interior  of  Ftx^], 


there  exists  a  closed  cone  Y  which  belongs  to  the. interior  of  F[x^]  and 
contains  y  In  Its  Interior.  Consider  a  sequence  of  points  x^  -♦  x^ 

(l  =  2,5,4,  .. .),  such  that  c^(x^  -  x^)  -♦  y,  (c^  >  O).  We  shall  prove 

that  X.  €  Q  Is  Impossible  (and  there  y  e  T  [Q]  Is  Impossible).  In  fact, 

^  *1 
c^(Xj^  -  x^)  T*  y  and  y  Is  interior  to  the  cone  Y,  therefore  x^  -  x^  e  Y 
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for  sufficiently  large  1,  and  by  Theorem  6.3  Is  reatchable  from 
for  1  larger  than  some  1^. 

§7.  Relations  with  the  boundary  of  R. 


Theorem  7.1*  Given  the  differential  equations  (I.5)  satisfying  the  condi¬ 
tions  stated  there,  denoting  by  R  =  the  reachab2.e  set  from  the  point 
Xq,  Sr  Its  boundary,  and  x^^  £uiy  polSt,  the  Intersection  of  the  cones 
F[Xj^]  and  [5R]  Is  void  or  belongs  to  the  boundary  of  F[x^]. 

The  proof  Is  Innedlate  If  we  apply  the  corollary  6.1,  taking  Into 
account  that  by  Theorem  3.3, 


T  [aR]  =  T^  [aQ]  C  T  [Q] 
*1  *1  *1 

where  Q  Is  the  complement  of  R. 


Theorem  7.2;  If,  under  the  saoie  assumptions,  the  trajectory 
to  the  boundary  of  R  for  t^  S  t  S  tg,  and 
x(t^)  =  x^,  the  cones  F[xj^]  and 


[an] 

1 


to  s  t.^  <  tg, 

have  a  ndn-vold 


x(t)  belongs 
then  denoting 
Intersection, 


which  belongs  to  the  bowdary  of  F[x^]. 

Taking  Irto  account  the  preceding  theorem.  It  remains  only  to  prove 

the  existence  of  at  least  one  common  ray  of  T  [aR]  and  F[x-].  This  ray 

*1 

Is  easily  obtained  by  taking,  on  the  trajectory  x(t),  a  decreasing  seqxience 
t^  -♦t^  (i  =  3,^,5,  .. .)  and  the  corresponding  points  x^^  =  x(t^)  e  aR. 


From  the  sequence 


°1^*1  -  *1^* 


1 


we  take  a  converging  svibsequence;  Its  limit  belongs  to  F[x^]  by  Lemma  6,1 

and  a  reasoning  similar  to  that  of  Theorem  6.1.  Therefore  ye  F[x^], 

y  €  T„  [aR];. 

'1 
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From  here  we  get  the  relation  with  Pontrjagin's  "Maximum  Principle". 
Indeed,  this  principle  applies  to  optimization  problems  characterized  pre¬ 
cisely  by  the  fact  that  any  optimal  trajectory  (i.e.,  solution  of  the  opti¬ 
mization  problem)  lies  on  the  boundary  of  the  reachable  set  (from  the 
initial  point).  Any  such  trajectory  has  a  definite  tangent  vector  at  almost 
every  point  and  by  Theorem  (7-2)  this  tangent  vector,  given  by  f(x(t),  u(t)), 
belongs  to  the  boundary  of  F[x(t)].  The  corresponding  value  of  u(t)  may 
or  itoy  not  belong  to  the  boxmdary  of  U,  this  fact  depending  on  the  mapping 
of  U  into  f(x,  U).  In  einy  case,  since  F[x]  is  convex,  there  exists  a 
vector  p(t)  such  that  u(t)  maximizes  the  scalar  product  p(t)*f(x(t),  u(t)) 
with  respect  to  all  possible  values  of  u  €  U,  and  this  IS  the  maximum 
principle. 

If  along  the  trajectory  the  boundary  of  the  reachable  set  R  has  a 
tangent  plane,  then  Theorem  5.2  shows  that  p(t)  satisfies  the  adjoint 
variational  equation  (5.IO). 

If  at  some  point  of  the  trajectory  the  boundary  of  R  has  a  tan¬ 
gent  plane.  Theorem  5.2  shows  that  the  vector  p(t)  satisfying  the  maxlmm 
principle  may  be  taken  to  satisfy  the  adjoint  variational  equation  even  in 
the  case  when  for  prevloxis  vadues  of  the  time  the  trajectory  goes  along  an 
(r-dimensional)  edge  of  the  boundary  of  R. 

The  main  difference  between  the  approach  of  Pontrjagln  and  this  paper, 
is  that  here  we  tried  to  use  global  properties  of  the  reachable  set,  while 
Pontrjagln  uses  small  variations  of  the  optimizing  trajectory. 

A  nice  picture  of  the  reachable  set  R  is  obtained  by  interpreting 
the  previous  results  in  terms  of  wave  propagation.  We  imagine 
x^  =  (t^,  x^)  as  the  origin  of  perturbations  propagated  along  all  admis¬ 
sible  trajectories.  Then,  for  any  fixed  value  of  t  >  t^,  the  section  of 
R  is  the  perturbed  regiort.  The  section  of  the  boundary  of  R  is  the  wave 
front.  The  boundary  of  R,  imagined  as  a  surface  given  by  an  equation 

t  =  S(x) 
min 

is  an  envelope  of  the  cones  F[x]  and  may  be  Interpreted  as  a  (generalized) 
solution  of  a  partial  differential  equation  of  the  first  order,  with  F[x] 
the  characteristic  cone  at  each  point.  This  clearly  points  out  the  relaticn 
with  the  Hamilton -Jacobi  theory. 
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